Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks

Parameter estimation becomes difficult in high-dimensional spaces due to the increasing sparseness of the data. Therefore, when a low-dimensional representation is embedded in the data, dimensionality reduction methods become useful. One such method-projection pursuit regression (Friedman and Stuetzle 1981 (PPR)-is capable of performing dimensionality reduction by composition, namely, it constructs an approximation to the desired response function using a composition of lower dimensional smooth functions. These functions depend on low-dimensional projections through the data. When the dimensionality of the problem is in the thousands, even projection pursuit methods are almost always overparameterized, therefore, additional smoothing is needed for low variance estimation. Exploratory projection pursuit (Friedman and Tukey 1974; Friedman 1987) (EPP) may be useful in these cases. It searches in a high-dimensional space for structure in the form of (semillinear projections with constraints characterized by a projection index. The projection index may be considered as a universal prior for a large class of problems, or may be tailored to a specific problem based on prior knowledge. In this paper, the general form of exploratory projection pursuit is formulated to be an additional constraint for projection pursuit regression. In particular, a hybrid combination of supervised and unsupervised artificial neural network (ANN) is described as a special case. In addition, a specific projection index that is particularly useful for classification (Intrator 1990; Intrator and Cooper 1992) is introduced in this context.